Apparatus and method for recognizing driving lane based on multiple sensors

ABSTRACT

An apparatus for recognizing a driving lane based on multiple sensors is provided. The apparatus includes a first sensor configured to calculate road information, a second sensor configured to calculate moving obstacle information, a third sensor configured to calculate movement information of a vehicle, and a controller configured to remove the moving obstacle information from the road information to extract only road boundary data, accumulate the road boundary data to calculate a plurality of candidate location information on the vehicle based on the movement information, and select final candidate location information from the plurality of candidate location information.

CROSS-REFERENCE(S) TO RELATED APPLICATIONS

This application claims priority to Korean Patent Application No.10-2020-0150903, filed on Nov. 12, 2020, which is incorporated herein byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates to an apparatus and a method forrecognizing a driving lane based on multiple sensors.

BACKGROUND

A technique for implementing a demonstration autonomous vehicle usingexpensive global navigation satellite systems (GNSS)/inertial navigationsystem (INS) is being studied. In particular, in the case of autonomousdriving, position recognition in a lateral direction is performedutilizing the INS and landmarks on a road, such as road markings, lanes,traffic lights, and road signs, using a camera.

However, in the case of the autonomous driving, position recognition ina transverse direction is important. To this end, a technology forsynthetically recognizing a driving lane and a position within a lane isrequired. In addition, in order to recognize the driving lane, roadboundary detection is required.

However, the road boundary detection has a problem of being inaccuratedue to weather, types of obstacles, and characteristics of sensors.

In addition, it may be undesirable to apply the GNSS/INS to autonomousvehicles for mass production.

SUMMARY

The present disclosure is directed to an apparatus and a method forrecognizing a driving lane based on multiple sensors, which may improveperformance of precise positioning on a road through a technique forroad boundary detection and driving lane recognition on the basis offusion of a low-cost global positioning system (GPS), multipleenvironment recognition sensors, and a precision map.

Other objects and advantages of the present disclosure can be understoodby the following description and become apparent with reference to someforms of the present disclosure. Also, it is obvious to those skilled inthe art to which the present disclosure pertains that the objects andadvantages of the present disclosure can be realized by the means asclaimed and combinations thereof.

In some forms of the present disclosure, there is provided an apparatusfor recognizing a driving lane based on multiple sensors, which improvesperformance of precise positioning on a road through a technique forroad boundary detection and driving lane recognition on the basis offusion of a low-cost GPS, multiple environment recognition sensors, anda precision map.

The apparatus for recognizing a driving lane includes a first sensorconfigured to calculate road information; a second sensor configured tocalculate moving obstacle information; a third sensor configured tocalculate movement information of a vehicle; and a controller configuredto remove the moving obstacle information from the road information toextract only road boundary data, accumulate the road boundary data tocalculate pieces of candidate location information on the vehicle on thebasis of the movement information, and select one among the pieces ofcandidate location information as final candidate location information.

In this case, the road boundary data may be extracted from a pluralityof previous frames using the movement information and accumulated in acurrent frame.

In addition, the pieces of candidate location information may becalculated using driving lane information obtained using a fourth sensoror precision map lane information which is set in advance on the basisof current vehicle location information.

In addition, the pieces of candidate location information may becalculated based on a grid formed by dividing a predetermined region ofinterest placed on a map road boundary of the precision map laneinformation at regular intervals.

In addition, the grid may be formed of a plurality of lateral directionsand a plurality of transverse directions, and all of the plurality oflateral directions and the plurality of transverse directions may be setas pieces of transverse candidate location information and pieces oflateral candidate location information.

In addition, the grid may be formed of a plurality of heading angles,the plurality of heading angles may be generated by dividing the regionof interest at regular intervals based on a center point of the vehicle,and all the plurality of heading angles may be set as candidate headingangle information.

In addition, the number of pieces of candidate location information maybe obtained by multiplying the number of pieces of lateral locationinformation, the number of pieces of transverse location information,and the number of pieces of heading angle location information.

In addition, the pieces of candidate location information may consist ofonly of a preset number of pieces of transverse candidate locationinformation based on the transverse direction in consideration of alocation error of the vehicle.

In addition, a location of the vehicle present within one road boundarymay be determined according to a ratio of a lateral offset calculatedbetween a driving lane and a center of the vehicle.

In addition, a preset number of the pieces of transverse candidatelocation information for each lane may be determined according to theratio of the lateral offset on the basis of a lane position and thenumber of lanes which are identified in a region of interest set inadvance in the precision map lane information.

In addition, a heading angle of each of a preset number of the pieces oftransverse candidate location information may be calculated using adifference value between a first average of a heading angle of a leftlane and a heading angle of a right lane in front of the vehicle, whichare recognized through the second sensor, and a second average of aheading angle of the left lane and a heading angle of the right lane ofa preset number of the pieces of transverse candidate locationinformation which are calculated through the precision map laneinformation.

In addition, the controller may match precision map road boundaryinformation searched on the basis of the pieces of candidate locationinformation with the accumulated road boundary data to assign a matchingdegree score to each of the pieces of candidate location information andselect a lane of the final candidate location information, to which ahighest matching degree score is assigned, as the driving lane.

In addition, the controller may convert the precision map road boundaryinformation and the accumulated road boundary data into a coordinatesystem formed of Y-axis coordinates and X-axis coordinates and compareonly values of the X-axis coordinates on the basis of an index of theY-axis coordinates to calculate the matching degree score.

In addition, the matching degree score may be calculated using thenumber of data points located in the vicinity of the precision map roadboundary information, the data points may be generated by the firstsensor, and the number of data points may be the number of left points,the number of right points, and the number of points between roadboundaries on the basis of the precision map road boundary information.

In addition, when the number of previous frames is larger than a presetvalue, the controller may remove one farthest previous frame and add onecurrent frame to keep an accumulated number of frames constant.

In addition, the accumulated number of frames may be variably setaccording to a vehicle speed or a movement distance of the vehicle.

In addition, the first sensor may be a distance sensor, the secondsensor may be a vision sensor, the third sensor may be a motion sensor,and the fourth sensor may be a global positioning system (GPS) sensor oran image sensor.

In addition, the controller may apply a preset invalid determinationcondition to the final candidate location information to determinewhether the final candidate location information is valid.

In addition, the invalid determination condition may include any oneamong a comparison condition whether the matching degree score of thefinal candidate location information is smaller than a preset firstmatching degree, a comparison condition whether a residual amount of thedistance sensor present in the driving lane is greater than a presetsecond matching degree in the final candidate location information, anda determination condition whether a left lane direction and a right lanedirection, which are in front of the vehicle, are parallel to each otheris determined according to a difference between the left lane directionand the right lane direction in the final candidate location informationand whether the vehicle deviates from a lane width by comparing adifference between a lateral offset of the left lane and a lateraloffset of the right lane based on the center of the vehicle with apreset road width.

In some forms of the present disclosure, there is provided a method ofrecognizing a driving lane based on multiple sensors, which includes afirst information calculation operation of calculating, by a firstsensor and a second sensor, road information and moving obstacleinformation; a second information calculation operation of calculating,by a third sensor, movement information of a vehicle; and a selectionoperation of removing, by a controller, the moving obstacle informationfrom the road information to extract only road boundary data,accumulating the road boundary data to calculate pieces of candidatelocation information on the vehicle on the basis of the movementinformation, and selecting one among the pieces of candidate locationinformation as final candidate location information.

In addition, the selection operation may include calculating, by thecontroller, the piece of candidate location information using precisionmap lane information preset on the basis of at least one of driving laneinformation obtained using a fourth sensor and current vehicle locationinformation obtained from a location information acquisition part.

In addition, the selection operation may include matching, by thecontroller, precision map road boundary information searched on thebasis of the pieces of candidate location information with theaccumulated road boundary data to assign a matching degree score to eachof the pieces of candidate location information and select a lane of thefinal candidate location information, to which a highest matching degreescore is assigned, as the driving lane.

In addition, the selection operation may include converting, by thecontroller, the precision map road boundary information and theaccumulated road boundary data into a coordinate system formed of Y-axiscoordinates and X-axis coordinates and comparing only values of theX-axis coordinates on the basis of an index of the Y-axis coordinates tocalculate the matching degree score.

In addition, the selection operation may include applying, by thecontroller, a preset invalid determination condition to the finalcandidate location information to determine whether the final candidatelocation information is valid.

DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an apparatusfor recognizing a driving lane based on multiple sensors in some formsof the present disclosure.

FIG. 2 is a detailed block diagram illustrating a configuration of acontroller shown in FIG. 1.

FIG. 3 is a flowchart illustrating a process of recognizing a drivinglane based on multiple sensors in some forms of the present disclosure.

FIG. 4 is a diagram for describing a concept of an operation ofseparating an output of a distance sensor shown in FIG. 3.

FIG. 5 is a diagram for describing a concept of the operation ofaccumulating an output of a distance sensor shown in FIG. 3.

FIG. 6 is a diagram for describing a concept of an operation ofgenerating candidate locations shown in FIG. 3.

FIG. 7 is a conceptual diagram illustrating an example of generating aheading angle candidate within a predetermined range according to FIG.6.

FIG. 8 is a conceptual diagram illustrating another example ofgenerating a candidate location according to FIG. 6.

FIG. 9 is a conceptual diagram for describing a search of a direction ofa recognized driving lane according to FIG. 8.

FIG. 10 is a conceptual diagram for describing a search of a headingangle of a precision map according to FIG. 8.

FIG. 11 is a flowchart of calculating a heading angle for each candidateposition according to FIGS. 8 to 10.

FIG. 12 is a diagram illustrating an example for describing a concept ofan operation of selecting an optimal candidate location shown in FIG. 3.

FIG. 13 is a diagram illustrating another example for describing theconcept of the operation of selecting an optimal candidate locationshown in FIG. 3.

FIG. 14 is a conceptual diagram illustrating an operation of setting amatching degree from a score distribution of correctly recognized finalcandidate location information and a score distribution of incorrectlyrecognized final candidate location information shown in FIG. 3.

FIG. 15 is a diagram for describing a concept of an operation ofdetermining invalidity shown in FIG. 3.

DETAILED DESCRIPTION

The present disclosure may be modified into various forms and may have avariety of forms, and, therefore, specific forms will be illustrated inthe drawings and described in detail. The forms, however, are not to betaken in a sense which limits the present disclosure to the specificforms, and should be construed to include modifications, equivalents, orsubstitutes within the spirit and technical scope of the presentdisclosure.

In describing each drawing, similar reference numerals are assigned tosimilar components. Although the terms “first,” “second,” and the likemay be used herein to describe various components, these componentsshould not be limited by these terms. The terms are used only for thepurpose of distinguishing one component from another component.

For example, without departing from the scope of the present disclosure,a first component may be referred to as a second component, andsimilarly, a second component may also be referred to as a firstcomponent. The term “and/or” includes a combination of a plurality ofrelated listed items or any item of the plurality of related listeditems.

Unless otherwise defined, all terms including technical or scientificterms used herein have the same meaning as commonly understood by thoseskilled in the art to which the present disclosure pertains.

General terms that are defined in a dictionary shall be construed tohave meanings that are consistent in the context of the relevant art andshould not be interpreted as having an idealistic or excessivelyformalistic meaning unless clearly defined in this disclosure.

Hereinafter, an apparatus and a method for recognizing a driving lanebased on multiple sensors in some forms of the present disclosure willbe described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating a configuration of an apparatus100 for recognizing a driving lane based on multiple sensors in someforms of the present disclosure. Referring to FIG. 1, the apparatus 100for recognizing a driving lane may include a controller 110, a locationinformation acquisition part 120 for acquiring current locationinformation of a vehicle, a sensor system 130 for calculatinginformation on a road on which the vehicle is traveling, information ona moving obstacle which moves on the road, and movement information ofthe vehicle traveling on the road, and a storage 140 for storing piecesof information. In addition, the apparatus 100 for recognizing a drivinglane may further include a power supply part 150 for supplying power tothe above components, and an output part 160 for outputting information.In addition, the components shown in FIG. 1 are mainly described forcomponents for the purpose of the apparatus for recognizing a drivinglane based on multiple sensors in some forms of the present disclosure,and other components are merely omitted.

The controller 110 transmits and receives signals to and from thecomponents 120 to 160 and performs a function of controlling thecomponents 120 to 160. In addition, the controller 110 performs afunction of removing the moving obstacle information from the roadinformation on the basis of the movement information to extract onlyroad boundary data, accumulating the road boundary data to calculatepieces of candidate location information on the vehicle, and selectingone among the pieces of candidate location information.

The location information acquisition part 120 performs a function ofacquiring current location information of the vehicle. Thus, thelocation information acquisition part 120 may be a global positioningsystem (GPS) receiver, but the present disclosure is not limitedthereto, and the location information acquisition part 120 may be aninertial measurement unit (IMU), a light detecting and ranging (LiDAR),or a radio detecting and ranging (RADAR). In addition, the IMU mayinclude an accelerometer, a tachometer, and the like.

The sensor system 130 may include a distance sensor 131 for sensing aroad on which the vehicle is traveling and generating distanceinformation on the road, a vision sensor 132 for generating a movingobstacle which moves on the same road, a motion sensor 133 for sensingmovement of the vehicle and generating movement information, and animage sensor 134 for sensing a lane of a road and generating drivinglane information.

The distance sensor 131 may be an ultrasonic sensor, an infrared sensor,a time of flight (TOF) sensor, a laser sensor, a LiDAR, or a RADAR.

The vision sensor 132 refers to a sensor for recognizing and evaluatinga moving object and a scene.

The motion sensor 133 refers to a motion recognition sensor forrecognizing a motion and a position of an object. Thus, the motionsensor 133 may include a sensor for detecting a direction, a sensor fordetecting movement, and a sensor for measuring a speed.

The image sensor 134 performs a function of capturing a lane on ageneral road. The image sensor 134 may be a charge-coupled device (CCD)sensor, a complementary metal-oxide semiconductor (CMOS) sensor, or thelike.

The storage 140 performs a function of storing a program having analgorithm for recognizing a driving lane based on multiple sensors,data, software, and the like. The storage 140 may include at least onetype of storage medium among a flash type memory, a hard disk typememory, a multimedia card micro type memory, a card type memory (e.g., asecure digital (SD) or extreme digital (XD) memory), a random accessmemory (RAM), a static random access memory (SRAM), a read only memory(ROM), an electrically erasable programmable read only memory (EEPROM),a programmable read only memory (PROM), a magnetic memory, a magneticdisk, and an optical disk.

The power supply part 150 performs a function of supplying power tocomponents. Thus, the power supply part 150 may be a battery pack formedof rechargeable battery cells, a lead acid battery, or the like.

The output part 160 performs a function of outputting a processingprocess of recognizing a driving lane to a screen. In addition, theoutput part 160 may also output processed data to the controller 110. Tothis end, the output part 160 may include a liquid crystal display(LCD), a light emitting diode (LED) display, an organic LED (OLED)display, a touch screen, a flexible display, a head-up display (HUD),and the like. The touch screen may be used as an input part. Inaddition, a sound system for inputting and/or outputting a voice, asound, or the like may be configured.

FIG. 2 is a detailed block diagram illustrating a configuration of thecontroller 110 shown in FIG. 1. Referring to FIG. 2, the controller 110may include a collection part 210 for collecting sensing data generatedfrom the sensor system 130, an accumulator 220 for accumulating roadboundary data extracted from a previous frame in a current frame usingthe movement information of the vehicle, a calculation part 230 forcalculating candidate locations of the vehicle to be placed in each laneusing driving lane information and precision map lane informationobtained on the basis of location information and for assigning scoresto the candidate locations to select a lane of a candidate location witha high score as a driving lane, and a determination part 240 forperforming a function of processing an exception for an unusualsituation which may occur in a real road.

The collection part 210, the accumulator 220, the calculation part 230,and the determination part 240 may be implemented in software and/orhardware so as to perform the above functions. The hardware may beimplemented with an application specific integrated circuit (ASIC)designed to perform the above functions, a digital signal processor(DSP), a programmable logic device (PLD), a field programmable gatearray (FPGA), a processor, a microprocessor, another electronic unit, ora combination thereof. Software implementation may include a softwareconfiguration component (element), an object-oriented softwareconfiguration component, a class configuration component and a workconfiguration component, a process, a function, an attribute, aprocedure, a subroutine, a segment of a program code, a driver,firmware, a microcode, data, database, a data structure, a table, anarray, and a variable. The software and the data may be stored in amemory and executed by a processor. The memory or the processor mayemploy various parts well known to those skilled in the art.

FIG. 3 is a flowchart illustrating a process of recognizing a drivinglane based on multiple sensors in some forms of the present disclosure.Referring to FIG. 3, the collection part 210 of the controller 110removes a surrounding vehicle corresponding to a moving obstacle from anoutput of the distance sensor 131 and extracts only road boundary data(S310). In other words, moving obstacle information is removed from roadinformation calculated by the distance sensor 131, and thus only roadboundary information is extracted. In order to generate the movingobstacle information, the vision sensor 132 may be used.

Thereafter, the accumulator 220 of the controller 110 accumulates theroad boundary data extracted from N (>1) previous frames in a currentframe using movement information of the vehicle obtained from an outputof the motion sensor 133 (S320).

Then, the calculation part 230 of the controller 110 searches transversecandidate location information of an own vehicle to be placed in eachlane using precision map lane information obtained on the basis ofdriving lane information acquired from the image sensor 134 and currentvehicle location information acquired from the location informationacquisition part 120 (S330).

The precision map lane information is stored in the storage 140 inadvance. Alternatively, the precision map lane information may besupplied from the outside through communication. That is, since alocation at which the vehicle is currently traveling is determined, thetransverse candidate location information is searched using latestprecision map information corresponding to the determined location. Tothis end, the precision map information may be stored in a database. Inaddition, the lane information and the road boundary information may beincluded in the precision map information.

In addition, the precision map lane information may be obtained on thebasis of both the driving lane information acquired from the imagesensor 134 and the current vehicle location information acquired fromthe location information acquisition unit 120 or may be selectivelyobtained on the basis of only one of the driving lane information andthe current vehicle location information.

Thereafter, the calculation part 230 of the controller 110 matches theprecision map road boundary information obtained on the basis of thecandidate location to an output of the road boundary informationaccumulated in S320 to give a score to each candidate location. As aresult, a lane of a candidate location with the highest score isselected as a driving lane. When the matching is performed at a positionof the driving lane, distance sensor data of a position close to aprecision map road boundary may be detected as a road boundary.

Thereafter, the determination part 240 determines whether selectedoptimal candidate location information is invalid using the function ofprocessing an exception with respect to a specific situation which mayoccur in an actual driving lane (S350).

Thereafter, the determination part 240 displays the results of detectingthe road boundary and recognizing the driving lane through the outputpart 160 (S360). The result information may be output as a combinationof graphics, a text, and a voice.

FIG. 4 is a diagram for describing a concept of the separating of theoutput of the distance sensor (S310) shown in FIG. 3. That is, FIG. 4 isa flowchart of extracting only a road boundary part of all pieces ofdata detected by the distance sensor 131 using the resultant recognizedby the vision sensor 132.

Referring to FIG. 4, a difference between the distance sensor data andvision sensor data is measured with respect to a moving object inadvance, modeled, and stored as a database in the storage 140.

Thereafter, the modeling result is reflected to a position and a size ofthe moving obstacle recognized by the vision sensor 132 and converted inthe form of an occupancy grid map. In this case, a value of 0 may beassigned to the moving obstacle and a value of 1 may be assigned toother areas.

Meanwhile, the output of the distance sensor 131 is also converted inthe form of an occupancy grid map. In this case, a value of 1 may beassigned to a sensor output position and a value of 0 may be assigned toother areas.

In a process of fusing the sensors, a road boundary map 430 iscalculated by performing an AND operation on a distance sensor map 410and a vision sensor map 420 in grid units. As a result, the movingobstacle is removed from a distance sensor output, and only a staticobject corresponding to the road boundary is obtained.

Alternatively, a distance sensor for separating a moving obstacle from astatic object and outputting the separation result may be used. That is,a plurality of distance sensors may be configured, and thus some of theplurality of distance sensors may be used to sense only moving obstaclesand the remaining thereof may be used to sense only static objects.

FIG. 5 is a diagram for describing a concept of the operation ofaccumulating an output of a distance sensor shown in FIG. 3. That is,FIG. 5 shows a concept in which pieces of data of a previous frame areaccumulated in a current frame. Referring to FIG. 5, the concept is asfollows.

1) The number of frames to be accumulated is set. In this case, a fixednumber of N (>0) may be set.

2) Location prediction (510) is required with respect to where the Npieces of previous frame data 502 are located in the current frame(i.e., a section). The location prediction may be calculated by applyingvehicle speed and angular velocity information (501), which is outputfrom the motion sensor 133, to a constant turn rate and velocity (CTRV)model. Generally, a frame is formed of a plurality of fields. In thiscase, previous frame data is placed in a data field of a current frame.

3) Accumulated road boundary data is obtained by accumulating roadboundary data of the current frame and pieces of road boundary data ofprevious frames (520 and 530). When the accumulation is performed, whenthe number of previous frames is greater than the set value of N, onefarthest previous frame is removed and the data of the current frame isadded to keep the accumulated number constant.

The road boundary data is increased through the above accumulation.

Meanwhile, the number of frames to be accumulated may be variably setaccording to a vehicle speed or a movement distance. In addition, whenthe number of frames is set to the vehicle speed, the accumulated numberis set to be smaller at a low speed and to be greater at a high speed.Since the low speed is mainly due to traffic jams, it is difficult tosufficiently secure the road boundary data due to surrounding vehicles.

FIG. 6 is a diagram for describing a concept of the operation ofgenerating candidate locations (S330) shown in FIG. 3. That is, FIG. 6is a conceptual diagram illustrating generation of a candidate locationon a three dimension, which consists of lateral/transverse locations andheading angle location information on the basis of a grid search.Referring to FIG. 6, a cross 610 indicates a center of an own vehicle, arectangle 600 indicates the own vehicle, and a square 620 including theown vehicle indicates a region of interest which is a range so as tosearch for a location of the own vehicle in lateral/transversedirections. In addition, the rectangle 600 indicates a location of theown vehicle, which is determined by GPS, a small dotted rectangle 611indicates a location of the own vehicle, which is determined byemergency vehicle preemption (EVP), and a small dotted triangle 612indicates a location of the own vehicle, which is determined by RT3002.Here, RT3002 is a model name of an intertial & GPS measurement system.

Generally, the location of the vehicle is determined bylateral/transverse locations and a heading angle. The set region ofinterest 620 is divided at regular intervals with respect to thelateral/transverse locations, all lateral/transverse locations generatedduring the division are set as candidate locations. For example, whenthe candidate locations are indicated as coordinates, the coordinatesmay be (50, 50), (−50, 50), (−50, 50), and (50, −50).

In addition, in FIG. 6, driving lane information 602 and map roadboundary 601 which are obtained using the image sensor 134 are shown.

FIG. 7 is a conceptual diagram illustrating an example of generating aheading angle candidate within a predetermined range according to FIG.6. Referring to FIG. 7, a set range (e.g., left and right sides ±10°) isdivided at regular intervals with respect to heading angle positioninformation, and all heading angles generated during the division areset as candidate heading angles. The number of generated candidatelocations becomes (the number of pieces of lateral locationinformation×the number of pieces of transverse location information×thenumber of pieces of heading angle location information).

The center of the own vehicle is obtained on the basis of a GPSposition, and a basic heading angle also becomes a heading angleobtained from the GPS. A range of the candidate locations may be set inconsideration of a GPS location error, and a search interval may be setaccording to an experiment in consideration of precision.

FIG. 8 is a conceptual diagram illustrating another example ofgenerating a candidate location according to FIG. 6. FIG. 8 is aconceptual diagram illustrating a setting of only a few transversecandidate locations using the precision map lane information and thedriving lane information recognized by the vision sensor 132. Referringto FIG. 8, a process of the setting is as follows.

1) A range of region of interest 800 in which candidate locations are tobe set in the transverse direction is set in consideration of the GPSlocation error. A size of the region of interest 800 may be about 20m×10 m. Although omitted from FIG. 8, precision map lanes (not shown)are present in the map road boundary 601.

2) A lateral offset between the driving lane recognized by the visionsensor 132 and the center of the own vehicle is calculated, and thelocation of the own vehicle positioned within one lane is determinedaccording to a ratio of the lateral offset.

3) Locations and the number of lanes entering the region of interest aredetermined at the precision map boundary obtained based on the GPS, andpieces of transverse candidate location information 811, 812, 813, and814 are searched at the ratio of the lateral offset for each lane.

FIG. 9 is a conceptual diagram for describing a search of a direction ofa recognized driving lane according to FIG. 8. FIG. 9 is a conceptualdiagram for describing a search of a direction of a driving lanerecognized by the vision sensor 132. Referring to FIG. 9, left and rightlanes L and R within a range from 5 to 10 m in a front side areobtained, each heading angle is calculated, and an average of the twoheading angles is calculated. As a result, the direction of the drivinglane recognized by the vision sensor 132 may be determined.

FIG. 10 is a conceptual diagram for describing a search of a headingangle of a precision map according to FIG. 8. FIG. 10 is a conceptualdiagram of a search of a lane direction in the precision map. Referringto FIG. 10, a precision map is obtained on the basis of the transversedirection value, the GPS lateral direction value, and the heading angleof each candidate location information, which are previously calculated,and then heading angles of the left and right lanes L and R arecalculated, and an average of the two heading angles is calculated. Theheading angles of the left and right lanes corresponding to eachcandidate position are obtained.

FIG. 11 is a flowchart of calculating a heading angle for each candidateposition according to FIGS. 8 to 10. Referring to FIG. 11, when adifference between the two heading angles obtained as the results ofFIGS. 9 and 10 is calculated, a GPS heading angle error is calculated,and when the GPS heading angle error is applied to the GPS headingangle, a heading angle of each candidate position is obtained.

The generation of the candidate locations is performed by searching thetransverse candidate locations for each lane and the heading angle ofeach candidate location. A location provided by the GPS may be directlyapplied as the lateral direction location.

FIG. 12 is a diagram illustrating an example for describing a concept ofan operation of selecting an optimal candidate location shown in FIG. 3.That is, FIG. 12 is a flowchart of searching for a location having thehighest matching degree to select an optimal candidate location byperforming matching between the road boundary of the precision map andthe road boundary of the distance sensor 131 on the basis of a distancetransform. Referring to FIG. 12, the concept is as follows.

1) A precise map road boundary 1201 is brought to a GPS referencelocation. Data 1210 is displayed in the form of an occupancy grid map.

2) When a binary image 1220 with respect to the precision map roadboundary is generated, a distance transform is applied to the binaryimage 1220.

3) Road boundary data 1230 of the distance sensor 131 is also brought inthe form of an occupancy grid map. Reference points of the two grid maps(the center of the vehicle) are matched, multiplication is performed ingrid units, and a distance transform-based matching result 1240 iscalculated. A matching score is calculated by calculating an average ofvalues assigned to the matched grid. As the matching score becomeshigher, the matching degree becomes higher. In particular, FIG. 12 showsthat a candidate location is searched on the basis of a distancetransform, and a result is obtained by detecting a precision map roadboundary 1203 and a distance sensor road boundary 1202 at a locationwith the highest matching degree.

4) In order to search for a candidate location, a reference point of adistance transformed image is moved to the candidate location which isset in S330 and the above process 3) is performed to calculate amatching score for each candidate location and search for the locationwith the highest matching degree.

FIG. 13 is a diagram illustrating another example for describing theconcept of the operation of selecting an optimal candidate locationshown in FIG. 3. FIG. 13 is a conceptual diagram illustrating that amatching degree between the road boundary of the precision map and theroad boundary data of the distance sensor 131 is calculated without theoccupancy grid map, and a candidate location having the highest matchingdegree is searched to select an optimal candidate location. Referring toFIG. 13, the concept is as follows.

1) All pieces of data are transformed in the form of an array havingY-axis (lateral axis) coordinate values sampled at regular intervals andX-axis (transverse axis) coordinate values corresponding to the Y-axiscoordinate values. In addition, front/rear/left/right regions ofinterest are set in advance based on the own vehicle.

2) The precision map road boundary data is brought to the candidatelocation, and the road boundary data of the distance sensor 131 is alsobrought so that the center of the own vehicle becomes the candidatelocation.

3) When only X-axis coordinate values of two pieces of datacorresponding to the sampled Y-axis coordinate index value are comparedso that a data point of the distance sensor 131 is positioned in thevicinity of the precision map road boundary, scores are calculated by asmuch as the number of points. For example, a road boundary point, aninput LiDAR point, a LiDAR point within a right road boundary range, anda LiDAR point within a left road boundary range are displayed.Therefore, a matching degree score is as follows.

Matching degree score=number of left points+number of rightpoints−lnumber of left points-number of right pointsl/2−number of pointsbetween road boundaries.

4) The matching degree score is calculated for each set candidatelocation to search for a candidate location with the highest matchingdegree.

5) A lane in which a candidate location with a high matching degree ispresent is recognized as a driving lane (1310). Matching degree scoresare given to the candidate locations, and the candidate locations arecompared with each other using the matching degree scores.

6) In order to detect the road boundary, an output of the distancesensor 131, which is present in the vicinity of the precision map roadboundary, is selected at the candidate location with the high matchingdegree (1320).

7) Since the selected data of the distance sensor is the dataaccumulated in S320, a plurality of points are placed at very closelocations. An average of adjacent points is calculated to remove excessdata.

8) Finally, in order to easily use the distance sensor data in whichonly the road boundary is detected, points having connectivity aregrouped into an object to be finally output (1330). That is, theaccumulated points are removed and only the road boundary of thedistance sensor data is detected.

In the output of the distance sensor 131 from which the moving object(that is, the moving obstacle) is removed, not only the road boundary issimply included, but also various noises, such as the moving object, andartificial structures, trees, and bushes outside the road boundary,which are not yet removed in S310, are included.

Therefore, in order to remove all the various noises and detect only apure road boundary or minimize an effect due to noise, when the distancesensor data is matched to the precision map, distance sensor data in thevicinity of the precision map road boundary is selected.

In addition, when a ratio of the number of left points is similar to aratio of the number of right points, a method of calculating thematching degree score may be designed to assign a higher score. Thescore is lowered by as much as the number of points remaining in theroad boundary so that it is possible to minimize an effect of the movingobject which is not yet removed.

FIG. 14 is a conceptual diagram illustrating an operation of setting amatching degree from a score distribution of correctly recognized finalcandidate location information and a score distribution of incorrectlyrecognized final candidate location information shown in FIG. 3.Referring to FIG. 14, an unusual situation such as a sensor outputerror, ambient noise, and a lane change may occur in a real roadsituation. In such an unusual situation, the technique for recognizing adriving lane in some forms of the present disclosure may not operatenormally. Accordingly, a detection result or a recognition result in anunusual situation may be processed as an exception through invaliditydetermination.

To this end, whether each frame is valid is output on the basis of apredetermined invalid determination condition. Three types of invaliddetermination conditions may be set.

Invalid Determination Condition 1:

When a matching degree score of finally selected final candidatelocation information is smaller than the matching degree, the finalcandidate location information is processed as being invalid. Each ofthe matching degree scores of the correctly recognized candidatelocation information generally has a value of zero or more (1410). Incontrast, a tendency of the matching degree score of the incorrectlyrecognized candidate location information, which has a value that issmaller than zero may be confirmed through a distribution map (1420).The matching degree may be set through a data distribution obtained fromthe results of some of the pieces of data.

Invalid Determination Condition 2:

When a residual amount of a sensor (e.g., LiDAR) present in the lane inthe finally selected final candidate location information is greaterthan the matching degree, the residual amount is processed as beinginvalid. The residual amount of the distance sensor 131 being largemeans misrecognition of the finally selected candidate. The matchingdegree may be set in the same manner as in the invalid determinationcondition 1.

Invalid Determination Condition 3:

When the driving lane recognized by the image sensor 134 is determinedas being invalid through a validity test, the driving lane is processedas being invalid. This will be described below with reference to FIG.15.

FIG. 15 is a diagram for describing a concept of an operation ofdetermining invalidity shown in FIG. 3. In particular, FIG. 15 shows amethod using the invalid determination condition 3. Referring to FIG.15, since a candidate position of the own vehicle is set using laneinformation recognized by the image sensor 134, when the resultrecognized by the image sensor 134 is invalid, an incorrectly recognizedresult is inevitably obtained. Therefore, a validity test of the drivinglane recognized by the image sensor 134 is the most importantdetermination condition for invalidity determination.

Whether lanes are parallel to each other 1510 is determined bycalculating directions of left and right lanes present within a rangefrom 5 m to 10 m in front of the own vehicle and determining adifference in direction between the left and right lanes.

Whether the own vehicle deviates from a road width 1520 is determined bycalculating lateral offsets of the left and right lanes based on thecenter of the own vehicle, calculating a difference between the lateraloffsets, and comparing the difference with a general road width d.

Consequently, it is possible to limit the use of a result of roadboundary incorrect detection or driving lane misrecognition due toerrors of the image sensor output and the distance sensor output throughthe function of determining invalidity. As a result, a driving lanerecognition rate is increased.

In addition, an error of the image sensor output mainly occurs during alane change. The error of the image sensor output directly affectsmisrecognition of the driving lane. Thus, a misrecognition result due tothe error of the image sensor output may be processed as exceptionthrough the invalid determination condition 3.

In addition, the output error of the distance sensor (especially, theLiDAR sensor) is unpredictable, and this also has a direct effect on themisrecognition of the driving lane. Thus, a misrecognition result due tothe error of the distance sensor output may be processed as exceptionthrough the invalid determination conditions 1 and 2.

The test results according to the invalid determination conditions 1 to3 are shown in the following table.

TABLE 1 Ratio of invalid frame 3.43% Driving lane recognition rate99.58% Processing time 32.9 ms

The processing time was obtained using a Matlab program of TheMathWorks, Inc.

Meanwhile, even in the case of a lane in which a distance sensor outputis hardly occurs, previous frames may be accumulated to maintain laneboundary information. That is, even when the lane boundary informationis lost by surrounding vehicles, a lane boundary may be detected byaccumulating previous lane boundary information.

In addition, the operations of the method or algorithm described in someforms of the present disclosure may be implemented in the form of aprogram command which is executable through various computer means, suchas a microprocessor, a processor, a central processing unit (CPU), andthe like, and may be recorded in a computer-readable medium. Thecomputer-readable medium may include program (command) codes, datafiles, data structures, and the like alone or in combination.

The program (command) codes recorded in the computer-readable recordingmedium may be specially designed and configured for some forms of thepresent disclosure or may be known and available to those skilled in thecomputer software. Examples of the computer-readable recording mediummay include magnetic media such as a hard disk, a floppy disk, amagnetic tape, and the like, optical media such as a compact disc readonly memory (CD-ROM), a digital versatile disc (DVD), a blu-ray disc,and the like, and semiconductor memory devices, which are specificallyconfigured to store and execute program (command) codes, such as aread-only memory (ROM), a random access memory (RAM), a flash memory,and the like.

Here, examples of the program (command) codes include machine languagecodes generated by a compiler, as well as high-level language codeswhich are executable by a computer using an interpreter or the like. Theabove-described hardware devices may be configured to operate as one ormore software modules so as to perform an operation of the presentdisclosure, and vice versa.

In accordance with the present disclosure, there is an effect in that,even when a shape of a road boundary is complicated or there is a lot ofnoises outside the road boundary or in a road, the road boundary can beaccurately detected through road boundary determination by matching witha precision map.

In addition, as another effect of the present disclosure, it is possibleto limit the use of a result of incorrect detection of the road boundaryor driving lane misrecognition due to errors of a camera sensor outputand a light detection and ranging (LiDAR) sensor output through afunction of determining invalidity so that a driving lane recognitionrate can be increased.

While the present disclosure has been described with reference to theaccompanying drawings, it will be apparent to those skilled in the artthat various changes and modifications may be made without departingfrom the spirit and scope of the present disclosure without beinglimited to some forms of the present disclosure. Accordingly, it shouldbe noted that such alternations or modifications fall within the claimsof the present disclosure, and the scope of the present disclosureshould be construed on the basis of the appended claims.

What is claimed is:
 1. An apparatus for recognizing a driving lane basedon multiple sensors, the apparatus comprising: a first sensor configuredto calculate road information; a second sensor configured to calculatemoving obstacle information; a third sensor configured to calculatemovement information of a vehicle; and a controller configured to:remove the moving obstacle information from the road information toextract road boundary data only; accumulate the road boundary data tocalculate a plurality of candidate location information on the vehiclebased on the movement information; and select final candidate locationinformation from the plurality of candidate location information.
 2. Theapparatus of claim 1, wherein the controller is configured to: extractthe road boundary data from a plurality of previous frames using themovement information and accumulated in a current frame.
 3. Theapparatus of claim 1, further comprising: a fourth sensor configured tocalculate driving lane information, wherein the controller is configuredto calculate the plurality of of candidate location information usingthe driving lane information obtained or precision map lane informationwhich is preset based on current vehicle location information.
 4. Theapparatus of claim 3, wherein the controller is configured to: form agrid by dividing a predetermined region of interest placed on a map roadboundary of the precision map lane information at regular intervals; andcalculate the plurality of candidate location information based on thegrid.
 5. The apparatus of claim 4, wherein the controller is configuredto: form the grid with a plurality of lateral directions and a pluralityof transverse directions; and set the plurality of lateral directionsand the plurality of transverse directions as a plurality of transversecandidate location information and a plurality of lateral candidatelocation information.
 6. The apparatus of claim 4, wherein thecontroller is configured to: form the grid with a plurality of headingangles; generate the plurality of heading angles by dividing the regionof interest at regular intervals based on a center point of the vehicle;and set the plurality of heading angles as candidate heading angleinformation.
 7. The apparatus of claim 6, wherein the controller isconfigured to: obtain the plurality of candidate location information bymultiplying the plurality of lateral location information, the pluralityof transverse location information, and the plurality of heading anglelocation information.
 8. The apparatus of claim 3, wherein the pluralityof candidate location information includes a preset number of transversecandidate location information based on the transverse direction inconsideration of a location error of the vehicle.
 9. The apparatus ofclaim 8, wherein the controller is configured to: determine a locationof the vehicle present within one road boundary according to a ratio ofa lateral offset calculated between a driving lane and a center of thevehicle.
 10. The apparatus of claim 9, wherein the controller isconfigured to: determine a preset number of transverse candidatelocation information for each lane according to the ratio of the lateraloffset based on a lane position and the number of lanes which areidentified in a region of interest preset in the precision map laneinformation.
 11. The apparatus of claim 9, wherein the controller isconfigured to: calculate a heading angle of each of a preset number oftransverse candidate location information using a difference valuebetween a first average of a heading angle of a left lane and a headingangle of a right lane in front of the vehicle, which are recognizedthrough the second sensor, and a second average of a heading angle ofthe left lane and a heading angle of the right lane of a preset numberof transverse candidate location information which are calculatedthrough the precision map lane information.
 12. The apparatus of claim3, wherein the controller is configured to: match precision map roadboundary information searched based on the plurality of candidatelocation information with the accumulated road boundary data; assign amatching degree score to each of the plurality of candidate locationinformation; and select a lane of the final candidate locationinformation having a highest matching degree score as the driving lane.13. The apparatus of claim 12, wherein the controller is configured to:convert the precision map road boundary information and the accumulatedroad boundary data into a coordinate system including Y-axis coordinatesand X-axis coordinates; and compare values of the X-axis coordinatesbased on an index of the Y-axis coordinates to calculate the matchingdegree score.
 14. The apparatus of claim 13, wherein the controller isconfigured to: calculate the matching degree score using a number ofdata points located in a vicinity of the precision map road boundaryinformation; and generate the data points by the first sensor, whereinthe number of data points includes a number of left points, a number ofright points, and a number of points between road boundaries based onthe precision map road boundary information.
 15. The apparatus of claim2, wherein the controller is configured to: when a number of previousframes is greater than a preset value, remove one farthest previousframe and add one current frame to keep an accumulated number of framesconstant.
 16. The apparatus of claim 15, wherein the controller isconfigured to: set variably the accumulated number of frames accordingto a vehicle speed or a movement distance of the vehicle.
 17. Theapparatus of claim 3, wherein: the first sensor includes a distancesensor, the second sensor includes a vision sensor, the third sensorincludes a motion sensor, and the fourth sensor includes a globalpositioning system (GPS) sensor or an image sensor.
 18. The apparatus ofclaim 17, wherein the controller is configured to: apply a presetinvalid determination condition to the final candidate locationinformation to determine whether the final candidate locationinformation is valid.
 19. The apparatus of claim 18, wherein the invaliddetermination condition includes any of a comparison condition whetherthe matching degree score of the final candidate location information isless than a preset first matching degree, a comparison condition whethera residual amount of the distance sensor present in the driving lane isgreater than a preset second matching degree in the final candidatelocation information, and a determination condition whether a left lanedirection and a right lane direction are parallel according to adifference between the left lane direction and the right lane directionin the final candidate location information, and whether the vehicledeviates from a road width by comparing a difference between a lateraloffset of the left lane and a lateral offset of the right lane based onthe center of the vehicle with a preset road width.
 20. A method ofrecognizing a driving lane based on multiple sensors, the methodcomprising: calculating, by a first sensor and a second sensor, roadinformation and moving obstacle information; calculating, by a thirdsensor, movement information of a vehicle; removing, by a controller,the moving obstacle information from the road information to extractonly road boundary data; accumulating, by the controller, the roadboundary data to calculate a plurality of candidate location informationon the vehicle based on the movement information; and selecting, by thecontroller, final candidate location information from the plurality ofcandidate location information.